Wearable electronic display and method for displaying information to a pilot

ABSTRACT

A wearable electronic display system detects aircraft flight related conditions, compares the detected flight related conditions with rules or procedures, retrieves flight condition information relating to phase of flight, and displays small amounts of contextually relevant flight condition information on a display screen.

FIELD

This invention relates to information displays for pilots and, morespecifically, to wearable information displays that present focused,situation appropriate information to the pilot in short messages.

BACKGROUND

Pilots operating aircraft are often presented with large amounts ofinformation in a short period of time. Most often, the information isprovided by the aircraft's instruments and radios. During high workloadphases of flight, the pilot may be presented with more information thancan be timely processed. As a result, pilots learn to prioritize certaininformation during certain phases of flight. For example, duringtakeoff, the pilot may prioritize engine, airspeed and attitudeinformation above all other types of information to ensure that theaircraft is placed in a condition for a safe takeoff. At other times,such as during abnormal or emergency situations, the pilot may becomeoverwhelmed with the amount of information presented by the aircraftinstruments.

Pilots currently deal with the high workload situations described aboveby prioritizing types of information (as discussed above) and/or bymemorizing short concise items, such as emergency checklists andoperating limitations. However, if a pilot is away from the flightstation, for example during rest periods, during physiological breaks,or during pre-flight activities, the aircraft information may not beavailable to the pilot beyond what the pilot has memorized. As a result,the pilot is not presented with real-time, prioritized information insuch circumstances.

SUMMARY

A wearable electronic display system detects aircraft and/or pilotconditions, compares the detected conditions with rules or procedures,retrieves contextual information relating to phase of flight or pilotphysiological condition, and displays small amounts of contextuallyrelevant information on a display screen.

The features, functions, and advantages that have been discussed can beachieved independently in various embodiments or may be combined in yetother embodiments further details of which can be seen with reference tothe following description and drawings.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is plan view of an aircraft, two different locations for pilots,and a wearable electronic display;

FIG. 2 is plan view of the aircraft of FIG. 1 including a close up viewof the flight deck;

FIG. 3 is a schematic diagram of a system for displaying small amountsof contextualized information on a wearable display;

FIG. 4 is a logic diagram followed by the system of FIG. 3;

FIG. 5 is a logic diagram for prioritizing information that is followedby the system of FIG. 3;

FIG. 6 is a logic diagram for determining emergency information todisplay that is followed by the system of FIG. 3.; and

FIG. 7 is a close-up view of the display of the system of FIG. 3.

DESCRIPTION

In one example, which is described herein, a system includes a wearableelectronic display and method for displaying small amounts ofcontextualized information to a pilot on the wearable electronicdisplay. In this example, the wearable electronic display comprises awrist mounted display screen, such as a watch with a display screen.However, in other examples, the display screen may be worn or carried onother parts of the body.

In this example, a wrist mounted electronic display screen monitorsflight phase pertinent information and compares the flight phasepertinent information to rules and criteria, whether regulatory oroperational in nature, and automatically prioritizes contextualinformation for display on the wrist mounted electronic display screen.The prioritization of contextual information will be discussed furtherbelow. As a result, a pilot has access to the prioritized contextualinformation regardless of the pilot's location within the aircraft orproximity to the aircraft. Moreover, the system may monitor a pilot'sphysiological state and tailor the information presented on the displayaccording to the pilot's physiological condition to aid the pilot'scomprehension of the presented information.

Referring to FIG. 1 a system 100 is illustrated that includes a wearableelectronic display 200 for displaying small amounts of contextualizedinformation to a pilot 210 on the wearable electronic display 200. Thesystem 100 may include connections to certain systems within an aircraft110, and the wearable electronic display 200. The wearable electronicdisplay 200 is disposed in this example on a pilot's wrist 220, similarto a watch. In some embodiments the wearable electronic display 200 mayinclude time information to function as a watch in addition to anelectronic display that displays prioritized contextual information. Thewearable electronic display 200 may be carried by the pilot 210 when thepilot 210 is seated at a flight station 230 within the flight deck ofthe aircraft 110, or when the pilot 210 is away from the flight station230, such as when the pilot is performing pre-flight duties at alocation 240 outside of the aircraft, when the pilot is on a rest breakin a crew rest facility (not shown) outside of the flight station 230,or when the pilot is on a physiological break, such as in the lavatory(not shown). Regardless of the pilot's location, the pilot always hasaccess to any information displayed on the wearable electronic display200.

Turning now to FIG. 2, the wearable electronic display 200 may bewireless sly connected with one or more aircraft systems, such as, forexample, a flight management system (FMS) 240, an onboard navigationsystem (ONS) 250, and/or an electronic flight bag system 260, such as,for example the Mobile FliteDeck®, FliteDeck VFR®, and FliteDeck Pro®,electronic flight bag systems marketed by Jeppesen®. Of course any otherelectronic flight bag product may be wirelessly connected to thewearable electronic display 200. Wireless connections may include WiFior Bluetooth® wireless connections, or any other type of wirelessconnection. In some embodiments, multiple wearable electronic displays200 may be wirelessly connected to one another. For example one wearableelectronic display 200 may be worn by the captain and another wearableelectronic display 200 may be worn by the first officer. In this case,the wearable electronic displays 200 may communicate wirelessly with oneanother.

Turning now to FIG. 3, the system 100 may include the wearableelectronic display 200, a processor 202, a memory 204 operativelyconnected to the processor 202, and a software program 206 stored in thememory 204 and executable on the processor 202. In the embodimentillustrated in FIG. 3, the processor 202, memory 204, and the softwareprogram 206 are all disposed within the wearable electronic display 200.In other embodiments, one or more of the processor 202, memory 204, andsoftware program 206 may be located outside of the wearable electronicdisplay 200 as long as the processor 202, memory 204, and softwareprogram 206 are communicatively connected to the wearable electronicdisplay 200, such as by a wireless connection described above.

The wearable electronic display 200 may also be operatively connected toone or more sensors, such as a physiological sensor 212, a positionsensor 214, and an orientation detector 216. The sensors provideinformation to the wearable electronic display 200 (more specifically tothe processor 202) that may be used to determine a phase of flight, anaircraft and/or a pilot condition, and an aircraft position andorientation. This information may then be used by the processor tofilter aircraft information and to prioritize the filtered informationfor display on the wearable electronic display 200.

The physiological sensor 212 may sense pilot related information suchas, for example, heart rate, blood pressure, eye movement, skinmoisture, skin temperature, etc., to determine the physiological stateof the pilot. For example, the sensed information may indicate that thepilot is highly stressed due to elevated heart rate and blood pressure.

The position sensor 214 may sense the spatial location of the pilot, insome cases relative to the spatial location of the aircraft 110, orrelative to a spatial location within the aircraft 110, to determinewhether the pilot is in the flight station 230. In one example, theposition sensor 214 may be a Global Positioning System (GPS) receiver,which locates the pilot's position in space independently of theaircraft position. In other embodiments, other types of position sensors214 may be used, such as radio frequency locators, magnetic locators, orvirtually any other type of locator. If the pilot is not located at theflight station 230, the system 100 may prioritize information that wouldbe relevant given the pilot's actual location. For example, if the pilotis located outside of the aircraft 110, the system 100 may prioritizeinformation relevant to pre-flight activities, such as maintenanceinformation like deferred or active maintenance items, flight planninginformation like fuel loads, air traffic control information likeestimated departure times, or any other information that may be relevantduring pre-flight operations. In contrast, if the pilot is locatedwithin the aircraft 110, but not at the flight station 230, the system100 may prioritize flight phase information, such as top of descentpoint, step climb points, areas of turbulence or icing, or emergency orabnormal situations, for display on the wearable electronic display 200.

The orientation detector 216 may sense the orientation of the aircraft110 relative to a reference plane, such as horizontal, vertical, or anyother reference plane. In some embodiments, the orientation detector 216may be part of existing aircraft systems, such as the FMS 240 or theattitude indicator. The orientation detector 216 may detect, forexample, aircraft pitch, aircraft yaw, aircraft roll, aircraft flightpath vector, aircraft speed, aircraft sink rate or climb rate, or anyother measure of aircraft orientation. The system 100 may prioritizeinformation that would be relevant given the aircraft's orientation. Forexample, if both aircraft pitch and flight path vector are positive(i.e., above the horizontal plane), then the system 100 may determinethat the aircraft is in a climb phase of flight and prioritizeinformation for the climb phase. If the aircraft pitch is positive andthe flight path vector is zero, the system 100 may determine that theaircraft is in a cruise phase of flight and prioritize information forthe cruise phase. If both the aircraft pitch and the flight path vectorare negative, the system 100 may determine that the aircraft 110 is in adescent phase of flight and prioritize information for the descentphase. If the aircraft pitch is positive, but the flight path vector isnegative, the system 100 may determine that the aircraft is in anapproach phase of flight and prioritize information for the approachphase. If the aircraft pitch is positive and very large (e.g., more than10 degrees) and the flight path vector is negative, the system 100 maydetermine that the aircraft is in a stalled condition and prioritizeinformation for the abnormal stall phase. Alternatively, the system 100may obtain phase of flight information from the FMS 240.

As illustrated in FIG. 3, the wearable electronic display 200 may bewirelessly connected to the onboard navigation system (ONS) 250, anelectronic flight bag (260), and/or a web server 270. Each of thesesystems may provide relevant contextual information to the processor 202so that the processor 202 may prioritize information for display on thewearable electronic display 200.

Turning now to FIG. 4, a logic diagram 300 is illustrated that may beused by the processor 202 (by executing for example the software program206) to determine contextually appropriate information to display on thewearable electronic display 200. In a first step 310, the processor 202may detect one or more flight and/or pilot related conditions (aircraftor physiological conditions) from one or more sensors. For example, theprocessor 202 may detect aircraft conditions from a trajectory sensor312 (for example the FMS or attitude indicator), the position sensor 214(for example a GPS receiver or an ONS device), a phase of flight sensor316 (which may be a combination of other sensors, for example, bycombining trajectory information and position information, or anindependent phase of flight sensor like the FMS), a proximity sensor 318(which may be a terrain avoidance system, or a radar altimeter system),the physiological sensor 212 (such as a heart rate sensor, a bloodpressure sensor, an eye movement sensor, a skin moisture sensor, or anyother physiological sensor), and an aircraft state or configurationsensor 322 (which may be the FMS or other aircraft configuration sensor,such as gear and flap indicators).

At step 340, the processor 202 may compare the sensed conditions(aircraft and/or physiological) against known rules and criteria. Forexample, if the processor 202 detects a clean aircraft configuration(i.e., landing gear and flaps are up) with an aircraft position that hasa low altitude above the ground (e.g., below 2,000 feet), the processormay prioritize configuration information for display on the wearableelectronic display 200. More specifically, in this case, the processor202 may direct the wearable electronic display 200 to display a messageinstructing the pilot to configure the aircraft for landing. Thelocation of the aircraft may also have a bearing on the context of theaircraft's position above the ground. The location of the aircraft maybe obtained at this step from the ONS 250.

At step 360, the processor 202 retrieves flight condition informationfrom the sensors and systems described above to aid in prioritizing thetype and amount of information to be displayed on the wearableelectronic display 200. The processor 202 may retrieve contextualinformation from the ONS 260, the electronic flight bag 260, and/or aweb server 270. The relevant contextual information may includeoperational procedures specific to a certain operating certificate (suchas the minimum altitude for configuring the aircraft for landing), orregulatory procedures that apply to all aircraft within a givenoperational jurisdiction (such as maximum airspeed below 10,000 feet).This contextual information is used to prioritize the information to bepresented on the wearable electronic display 200.

At step 370, the processor 200 prioritizes the flight conditioninformation for display on the wearable electronic display 200. Theprioritization process is described further below with respect to FIG.5.

At step 380, the processor 200 sends the prioritized flight conditioninformation to the wearable electronic display 200 in small packets sothat the presented information is quick and easy to read and understand.Generally, small packets of information are limited to less than 100characters, and in some cases less than 50 characters, depending uponthe type of information to be displayed.

Turning now to FIG. 5, the prioritization process is illustrated indetail. Initially, at step 410 the processor 200 determines whether thedetected condition from step 310 in FIG. 3 is related to an emergency bycomparing characteristics of the detected condition to indications of anemergency condition that may be stored in the memory 204. Emergencyconditions may be set by the aircraft manufacturer, which may bereflected in the quick reference handbook (QRH) or other operationalmanual. If desired, emergency conditions may be set by the aircraftoperator to include more conditions than those identified by theaircraft manufacturer. Regardless, if the processor 200 determines thatthe detected condition is related to an emergency condition (forexample, engine failure, engine fire, engine stall or surge, rapiddepressurization, or any other determined emergency condition), theprocessor 200 then sends emergency related information (such as anemergency checklist or memory item), which also may be retrieved fromthe memory 204, to the display at step 412. A memory item is a portionof the emergency checklist that includes immediate action items, whichare often required to be memorized by the pilots as part of operationsspecifications. The process of determining the emergency information tobe displayed will be discussed further with respect to FIG. 6.

If the processor 200 determines that the detected condition is notrelated to an emergency condition, the processor 200 proceeds to step414 to determine if the detected condition is related to an aircraftoperational limitation (such as maximum or minimum airspeed, maximumflap extension speed, turbulence penetration speed, or any otheraircraft operational limitation) by comparing characteristics of thedetected condition to indications of an aircraft operational limitationthat may be stored in the memory 204. If the processor 200 determinesthat the detected condition is related to an aircraft operationallimitation, the processor 200 then proceeds to step 416 to determine ifemergency information is displayed from step 412. If the processor 200determines that emergency information is already displayed, theprocessor then proceeds to step 418 to determine the pilot's stresslevel by, for example, retrieving physiological information from thepilot by means of the physiological sensor 212. If the processor 200determines that the pilot is showing signs of stress, by comparinginputs from the physiological sensor 212 to known indications of stressthat may be stored in the memory 204, the processor 200 inhibits displayof the aircraft operational limitation at step 420. The purpose of thisstep is to display only the most critical information to the pilot sothat the pilot does not become cognitively overloaded in emergency orabnormal flight situations. If, on the other hand, the processor 200determines that the pilot is not showing signs of stress, the processor200 may send the aircraft operational limitation information to thedisplay at step 422.

If the processor 200 determines that the detected condition is notrelated to an emergency condition or an aircraft operational limitation,the processor 200 then determines if the detected condition is relatedto a regulatory limitation (such as maximum airspeed below 10,000 ft. inthe U.S.), at step 430 by comparing characteristics of the detectedcondition to indications of a regulatory limitation that may be storedin the memory 204. If the detected condition is related to a regulatorylimitation, the processor 200 then determines if there is emergencyinformation or aircraft operational limitation information displayed atstep 432. If there is emergency information or aircraft operationallimitation information displayed, the processor 200 then determines ifthe pilot is showing signs of stress at step 434. If the pilot isshowing signs of stress, the processor 200 inhibits display of theregulatory limitation at step 436. If the pilot is not showing signs ofstress, the processor 200 may send the regulatory limitation informationto the display at step 438.

If the processor 200 determines that the detected condition is notrelated to an emergency condition, to an aircraft operationallimitation, or to a regulatory limitation, the processor 200 thendetermines if the detected condition is related to a flightnon-emergency condition (such as approaching top of climb or top ofdescent point on the flight plan) at step 440, by comparingcharacteristics of the detected condition to indications of a flightnon-emergency condition that may be stored in the memory 204. If thedetected condition is related to a flight non-emergency condition, theprocessor 200 then determines if there is emergency information,aircraft operational limitation information, or regulatory limitationinformation displayed at step 442. If there is emergency information,aircraft operational limitation information, or regulatory informationdisplayed, the processor 200 then determines if the pilot is showingsigns of stress at step 444. If the pilot is showing signs of stress,the processor 200 inhibits display of the regulatory limitation at step446. If the pilot is not showing signs of stress, the processor 200 maysend the regulatory limitation information to the display at step 448.

If the processor 200 determines that the detected condition is notrelated to an emergency condition, to an aircraft operationallimitation, to a regulatory limitation, or to flight non-emergencyinformation, the processor 200 then determines if the detected conditionis a communication (such as a short message from another pilot), at step450. The processor 200 may send the communication to the display at step452 regardless of what other information is displayed because crewcommunication is of primary importance at all times.

Turning now to FIG. 6, prioritization of emergency information isdescribed. The prioritization steps illustrated in FIG. 6 may take placeafter the processor 200 has determined that the condition is related toan emergency at step 410 in FIG. 5. At step 510, the processordetermines if an emergency checklist exists for the detected condition,by comparing characteristics of the emergency condition to a list ofemergency checklists that may be stored in the memory 204. For example,an emergency checklist may be any checklist contained in the QRH orother operational manual. If no emergency checklist exists, theprocessor 200 may send a description of the emergency condition to thedisplay at step 512. If an emergency checklist exists, then theprocessor 200 determines if the emergency checklist includes memoryitems at step 520. If no memory items exist, the processor 200 may senda short description of the emergency checklist (such as title and pagenumber in the QRH) to the display at step 522. If the processor 200determines that memory items exist, the processor 200 may send thememory items to the display at step 530. The memory items may not belimited by the normal short message limitation of 100 characters orless, but rather, the processor 200 may send the entire set of memoryitems to the display to ensure that the pilot has access to all thememory items.

Turning now to FIG. 7, one example of a wearable electronic display 200is illustrated. The wearable electronic display 200 may include adisplay screen 201, a case 203 that supports and protects the displayscreen 201, and one or more operator interface devices, such as buttons205. The display screen 201 may be divided into areas or sectors for thepresentation of certain information. For example, a first sector 207 maybe reserved for flight informational parameters, such as suggested topof climb or top of descent points, weather information, or other flightrelated information. A second sector 208 may be reserved for timecritical, very important information, such as memory items or emergencychecklists. A third sector 209 may be reserved for short messages, forexample from another pilot or from the dispatcher. Each sector 207, 208,209, may include a defined number of characters for display. Forexample, the first sector 207 may be limited to 50 characters, or less,the second sector 208 may be limited to 100 characters or less, and thethird sector 209 may be limited to 50 characters or less. In this way,the pilot immediately knows the level of importance of the presentedinformation simply based on the location of the information on thedisplay screen 201 and the amount of information presented in eachsector 207, 208, 209 is limited for ease of comprehension whilebalancing the need to display enough information to be helpful to thepilot.

In some embodiments, the wearable electronic display 200 may includealerting mechanisms, such as haptic/tactile alarms 211 (such as avibrating mechanism) or audible alarms 213 (such as a speaker) to alertthe pilot that relevant contextual information is being displayed.

The system and wearable electronic display described aboveadvantageously present a pilot with real time, contextually relevantinformation in small pieces that are easy to read and comprehend,regardless of the pilot's location within or outside of the aircraft andregardless of the pilot's physiological state.

While various embodiments have been described above, this disclosure isnot intended to be limited thereto. Variations can be made to thedisclosed embodiments that are still within the scope of the appendedclaims.

What is claimed is:
 1. A system for displaying contextualized flightinformation to a pilot on a wearable electronic display, the systemcomprising: a wearable electronic display; a processor operativelycoupled to the wearable electronic display; a memory operatively coupledto the processor; and a software program stored in the memory andexecutable on the processor, the software program detecting a flightrelated condition, retrieving flight condition information related tothe flight related condition, comparing the flight related conditionagainst rules and criteria, and using the flight condition informationto prioritize contextual information related to the flight relatedcondition that is displayed on the wearable electronic display.
 2. Thesystem of claim 1, wherein the prioritized contextual information islimited to 100 characters or less.
 3. The system of claim 1, wherein theprocessor is communicatively connected to, and retrieves the flightcondition information from, one of an aircraft flight management system,an electronic flight bag system, a proximity locator, and an orientationdetector.
 4. The system of claim 1, wherein the processor compares theflight related condition to an aircraft limitation parameter.
 5. Thesystem of claim 4, wherein the aircraft limitation parameter is anaircraft operational limitation.
 6. The system of claim 5, wherein theaircraft limitation parameter is an aircraft regulatory limitation. 7.The system of claim 5, wherein the flight condition information isretrieved from a flight management system.
 8. The system of claim 5,wherein the flight condition information is retrieved from an electronicflight bag.
 9. The system of claim 1, wherein the prioritized contextualinformation includes at least one of: takeoff minima, approach minima,missed approach information, top of climb location, top of descentlocation, radio frequencies, holding information, deicing holdovertimes, and short messages from dispatch or other pilots.
 10. The systemof claim 1, wherein the wearable electronic display includes an alarm toalert the pilot to the contextual information that is displayed on thewearable electronic display and, wherein the alarm includes one of ahaptic alarm, a tactile alarm, a vibration alarm, and an audible alarm.11. The system of claim 1, wherein the prioritized contextualinformation is prioritized based on the location of the wearableelectronic display relative to the flight station.
 12. A wearableelectronic display, the display comprising: a display screen; and aprocessor operatively coupled to the display screen; wherein theprocessor is programmed to display, on the display screen, small amountsof flight condition information after comparing the flight conditioninformation against rules and criteria, and to prioritize the flightcondition information based on a phase of flight to which the type offlight condition information is related and based on a location of thewearable electronic display.
 13. The display of claim 12, wherein theflight condition information includes at least one of: takeoff minima,approach minima, missed approach information, top of climb location, topof descent location, radio frequencies, holding information, deicingholdover times, and short messages from dispatch or other pilots. 14.The display of claim 12, wherein the processor compares the flightcondition information to an aircraft limitation parameter.
 15. Thesystem of claim 14, wherein the aircraft limitation parameter is anaircraft operational limitation.
 16. The system of claim 15, wherein theaircraft limitation parameter is an aircraft regulatory limitation. 17.The system of claim 15, wherein the flight condition information isretrieved from a flight management system.
 18. The system of claim 15,wherein the flight condition information is retrieved from an electronicflight bag.
 19. The display of claim 12, wherein the flight conditioninformation is limited to 100 characters or less.
 20. A method ofdisplaying flight condition information on a wearable electronicdisplay, the method comprising the steps of: detecting one or moreconditions of an aircraft; comparing the detected conditions to knownrules and criteria; retrieving contextual information relating to one ormore detected conditions; prioritizing the contextual information basedon the one or more detected conditions; and displaying small amounts ofprioritized contextual information on the wearable electronic displayscreen, wherein the known rules and criteria include one or more of anaircraft operational limitation and a regulatory limitation.